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ABSTRACT 

Three facets of literacy measurement were examined in 
this study: the assessment of functional literacy with an instrument 
designed to reflect practical reading tasks; the effect of mode of 
test administration on literacy; and the relationship of literacy to 
sex, schooling, and traditional measures of reading. Subjects were 
626 young adults enrolled in four Jol '"orps residential centers, 
R/EAL (Reading/Everyday Activities in Life), a new test of functional 
literacy, was administered to subjects under one of three conditions: 
English-taped, English-printed, or Spanish-taped, Results indicated 
that under each of the conditions, R/EAL proved to be a reliable and 
valid measure of literacy. Significant relationships were found 
between literacy performance and number of years of school completed 
and between literacy and traditional reading for each condition. No 
significant relationships were found between sex or length of stay in 
program and literacy. Conclusions are that literacy measurement on 
practical reading tasks can be accomplished, that test administration 
conditions do not significantly affect test performance for groups at 
comparable reading levels, and that a number of factors appear to be 
associated with such measurement* (Author/TO) 
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Three facets of literacy measurement were examined in this study: 
the assessment of functional literacy with an instrument designed to 
reflect practical reading tasks; the effect of mode of test administration 
on literacy; and the relationship of literacy to sex, schooling, and 
traditional measures of reading* Subjects included 626 young adults 
enrolled in four Job Corps residential centers. Job Corps is a 
federally supported program designed to teach 16-21 year old disadvantaged yout 
basic reading and math skills, job-oriented skills, and social living. 
R/lAL, a new test of functional literacy, was administered to subjects 
under one of three conditions: English-taped, English-printed or 
Spanish-taped. Data on years in school, time in program, and traditional 
reading measures were obtained from existing records. Results 
indicated that under each of the three conditions, R/EAL proved to be a 
reliable and valid measure of literacy. Anova by form of administration 
Indicated significant differences between English and Spanish taped formats 
but none between English .taped and printed conditions. However, these 
differences appeared to be due uore to initial differences in reading 
rather than form of administration. Significant relationships were 
found between literacy performance and nuniber of years of school completed, 
and between literacy aid traditional reading for each condition; yet there 
were no significant relationships between sex or length of stay in program 
and literacy. Conclusions are that literacy measurement on practical 
reading tasks can be accomplished, tliat test administration conditions do 
not significantly affect test parfovinauce for groups at comparable reading 

m 

levelE, and that a nuAjbe** cf factor-^ rt|^ear to be associated with svich 
uicasuretnfint . 
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Increasing public pressure for educational accountability has led 
to new definitions and increased asscasmcnc of reading throughout the United 
Stotea, Harris • surveys (1970, 1971) explored the extent of the literacy 
problem on the national scene ♦ In surveyinfi, the ability of adults to perform 
such reading tasks as locating information in ii telephone directory and 
completing a medicare form, Harris determined that as many as 18 k million 
Americans are functionally illiterate (1970/ p* 7) • Illiteracy rates 
broken out by sex, race, geographic location and other variables led Harris 
to observe that ''despite this sharp difference in results /Blacks scoring 
considerably lower than whites^/, we do not feel that race alone is a 
meaningful indicator of reading ability'' (1971, p. 54). Rather, he 
suggested that race is closely correlated with educational background* In 
his sample three times as many Blacks stopped their education at 8th gr^^de 
or earlier. Further, Harris concluded that income level also was an 
Indicator of reading ability. Anc in examining age, Harris noted "Since 
functional reading ability is closely linked with practical experience in 
real*-life situations, it seems reaconable that 16-year-o3<)3 scored lower 
than others interviewed" (1971*, p. 57). Difficulties among those over 
50 were also noted. These two suxrveys suggest the pervasive and wide- 
spread nature of the problem. More importantly, they offer new inter- 
pretations of reading and literacy, i.e., that it be concerned with 
applications of reading to practical material, 

A full scale determination of the reading habits of adult 
Americans and the subsequent development of adult tasks suitable for 
the assessment of literacy Ixave been undertaken by Educational Testing 
Service (Jackson, 1972 and Cm-von, 1972). Sharon's report derails the 
kinds of m<ite^tals adults rcicL 5,067 adultii repvr^Renling a cros*^'- 
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ecctlon of the population were queried regarding their reading habits. 

A sumrnary of their responses provides empirical support for the hypothesis 

that arlults tend to spend about one and three quatters hours per day 

reading newspapers, magazines, books, and job-related materials (Sharon, p. 16), 

Newspapers are rea^ more frequently than any other material, Jackson's 

work considers the development of adult tasks for literacy assessment. 

Detailed information on the nature of this pro^act has not been made 

available, ho^^ever. 

Somewhat related to the development of adult tasks for literacy 
assessment as proposed by Educational Testing Service is the work con- 
duct<:d by Sticht, Caylor and others (Sticht & Kern, 1970; Sticht, Caylor, 
Kern & Fox, 1971; and Sticht & Caylor, 1972), In a study conducted for 
the Army, Sticht ead Caylor explored "how well a person needs to read to 
perform selected adult reading tasks..." (1972, p. 29). In their view, 
reeding was often job related and literacy should be defined in terms of • 
the reading demands imposed by the job* Reading tasks were developed for 
specific jobs such as cook and repairman* In examining the relationship 
between performance on the Job Reading Task Tests and scores on a standard- 
ized reading test and the Armed Forces Qualifying Test, they noted that •'to 
a large extent these, three instruments appear to be measuring similar 
skills" (1972, p. 42), However, it was found that Job Reading Task Tests 
were more suitable in measuring ability to perform job related reading tasks. 
Tlicy concluded that literacy demands of the job should be considered in 
the development of adult basic education objectives and related assevimcnt 
procedures « 
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Building on the premise that assessment of reading should be 
task oriented, the National Assessment of Educational Progress developed 
series of tasks based on reading objectives which were used to assess 
reading performance of students at various. age levels, A large number 
of tasks were developed covering eight general categories (National 
Assessment, 1972a). Tasks were administered to samples of students at 
four age levels and performance based on median percentage of a group 
passing a given section was reported. Differences associated with sex, 
race, parental education, geographic region, and size and type of 
community were computed (National Assessment, 1972b), A recent National 
Assessment study supported by the Office of Education will determine 
literacy levels among 5200 17-year-olds attending school. This assess- 
ment utilizes exercises "based on practical reading material--advertise- 
ments, labels, television program schedules" (Report on Education Re- 
search, 1973, p. 3 and Wilson", 1973). In a somewhat related effort, Hanaen 
and Hesse (1974) have been attacking the assessment of literacy by utilizing 
cloze passages representing cross sections of ten content domains of 
reAding material including consumer, safety and occupational areas. 

The direction of current research lends support to the position 
that assessment of reading and functional literacy should focus on 
practical reading material reflective of adult reading habits, job re- 
lated skills , or educational objectives. Tied in with the practicality 
aspect is the need for the individual to demonstrate his ability to apply 
reading skills to concrete reading tasks. Meaningful and practical inter- 
pretation of reading performance can better be achieved if the test items 
arc directly related to predetermined tasks and objectives of literacy cr 
to job rclatcu tasks (Sticht and Caylor, 1972). 
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Other considerations related to reading or literacy test per- 
formance have also surfaced. One area of interest da the differential 
effect of test format on the test performance of individuals who Y.^ve 
experienced extreme difficulties in reading. For some time efforts have 
been directed at identifying optimum test administration conditions 
especially for pupils of low socio-economic status with low reading achieve- 
ment (Bennett & Doppclt, 1968; Ferris & Nichols, 1967, 1968; Ferris et al, 
1968; and Larsen & Rogers, 1968). Ferris and Nichols (1969) explored the 
test performance of third and seventh grade students under four condi- 
tions including individual interview, individual self -paced tape. 
Individual speaker-paced tape, and group speaker-paced tape. To their 
surprise, the data analysis revealed that for certain types of perfor- 
mance the individual interview and group speaker-paced proved signi- 
ficantly higher than the other two conditions. Hollenbeck and Micucci's 
work (1974) exploring effe«.ti5. of oral or written test format on test 
performance of black and white women found no significant effect due to 
' test format; likewise Hopkins et al (1967) found no effect of test 
administration format (television v. teacher) on the performance of 
fifth and sixth grade students. Further, Remer (1974) reported no 
difference in test performance due to test modality (videotape, audio- 
tape, and transcript). Lovins (1973) varied test administration format 
by establishing two conditions for the administration of an arithmetic 
test to sixth grade black children. Using cither a perfunctory, minimal, . 
concise instructional set or .VK Vi.t.h an atmosphere of kindness and empathy, 
Lovins found no sifinificant. ciiV{<tr«nccs due tc test condition.. Yet in 
spite of the conclusions one. mlc;ht rer.ch from the above research, Ferris 
and Nichols, in interpreting their study suggested that "/?Jor some reason, 
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perhaps novelty, response mode, isolation from classroom, individual 
tec ting in the van by self- and speaker-paced tape recordings elicited 
poorer test performance. j/^"han in the other conditions^* . (1969, p. 90). 
Further, although some of the studies dealt* with poor readers and those 
of low socio-econorfiic status, none was specifically aimed at the young 
adult whose accumulated failure in test taking situations may make the 
mode of test presentation an even greater factor affecting his test 
performance. 

Davis and Fersonke (1968) examined test performance as a 
function of test administration in English or Spanish. Using a sample 
of eighty-eight Spanish speaking young children, they found no difference 
in performance on the Metropolitan Readiness Test due to language in 
which the test was administered. However, they suggested caution in 
broad interpretations of this finding. Zintz (1970), in a considera- 
tion of reading instruction, suggested that studeuts will be able to 
read and write English better if first given an opportunity to speak it. 
An inability to speak English fluently may also have an effect on the stu- 
dent^s ability to process information presented orally in English. It 
may be that oral presentation in the nacive language may facilitate test 
performance. 

In addition to factors directly connected with the content, 
format and administration of a test, other factors not directly associated 
with the test per se may be related to literacy t^^st performance. One 
ouch factor specifically explored in this study mole-female differences. 
Althouj^h studies examining male-female difference In reading performance 
have found generally that females achieve at higher levels than males, 
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Uroff (1972) found that high school males made significantly greater 
gains than females on a measure of reading achievement growth and Long 
(1972) found that male high school students achieved higher than females 
in the math area. An apparent conflict in* the literature suggests 
further Investigation of the possible relationship between literacy 
performance and sex. 

Finally, certain aspects of the effects of schooling, whether 
number of years of school completed or amount of time in present school 
Situation, on literacy achievement warrant additional exploration. 
Stauffer (1973), in his extensive descriptive study of 96 adult 
literacy programs in 29 states, noted a positive relationship between 
number of years of formal schooling and reading gains, except in the 
case of white students. He also found positive relationships between 
length r.nd duration of reading class and reading gains. 

Concerns of the present study center on three aspects of the 
literacy assessment problem: the assessment of functional literacy with 
a reliable and valid instrument designed to reflect practical reading 
tasks ; the examination of the effect of mode of test administration on 
literacy performance; and the exploration of the relationship of literacy 
performance to sex, schooling, and traditional measures of reading per- 
formance . 

Method 

• . * 

Sample 

The subjects in this study consisted of 62G Job Corps enrol kc.y 
(339 females and 287 males), who were full time students irt tl;e proc^mm 

ERIC » 
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Job Corps is a federally supported program designed to teach 16-21 year old 
disadvantaged youth basic reading and math skills, job-oriented skills, 
and social living. Students were selected from one of four Job Corps 
Centers located in New Jersey, New Mexico, Texas, and West Virginia. For 
the most part students were Black and Mexican-American. A minority of 
students was white. Sauple selection varied from Center to Center due to 
conditions prevailing at the individual Centers; however, in most cases 
students were selected randomly. 

' Procedures 

Literacy performance was determined by R/EAL, a -''S-item criterion- 
referenced test consisting of nine reading selections, including road 
signs, want ads, a lease, an application form and other reading materials 
selected to represent common reading activities (Lichtman, 1972, and 1974, 
in press) . Detailed reading objectives and task analyses for each reading 
selection provide the user with concrete information about the reading 
tasks involved. An objective scoring key facilitates evaluation of 
responses. R/EAL yields a mastery level score which is defined as passing 
80 percent or more of the items, such mastery being indicative of functional 
literacy. 

Standardised reading achievement measures included the Stanford 
Achievement Test (Kellcy, et al., 1964), various forms and levels, and 
the RJSl Test, a 25-item screening instrument consisting of graded para- 
graphs with selected words deleted. Student selects the correct response 
from a set of . alternatives . This test, specifically developed for Job 
Cort>s, is used as an aid in placement of students and in determining 
suitable levels of the Stanford to administer. 

In order to explore the effect of test administration format on 
literacy performance, r/EAL was administered under one of three conditions: 
ERJC standard En^Uish taped, Standard English printed, and Sp/)nish taped. .Undijr 1" 
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the first and third conditions each student was tested using an individually 
operated and controlled audio cassette player with all directions and 
questions in English or Spanish. Under the second condition each student 
received all directions and questions in Standard English printed directly 
in the R/E^L test booklet. For each of the three conditions all material 
which the subject actually read was displayed in English in the R/EAL 
test booklet. All responses were '-ide in .writing (English or Spanish) 
directly in the test booklet. 

In Centers where no Spanish-speaking students were present, 
subjects were assigned to the first two conditions randomly. However, 
due to other requirements of the study many more students were tested 
under the first testing procedute. In Centers where Spanish -speaking 
students were present, subjects who were monolingual Spanish or bi- 
lingual were assigned to the third condition. Although no precise figures 
were available, about half of the subjects receiving the third condition 

were bilingual. 

The study was conducted on site in four geographic locations 
bv the author and members of the project staff during November and 
December 1972. All data on related variables, including Stanford 
Achievement Test and RJSl Test scores were collected during the on site 
visits from existing student records. 

Results 

Three aspects of literacy assessment were considered in this 
study: determination of literacy levels of subjects; effect of test mode 
on literacy pcrf ormarNoe ; and the examination of literacy performance as 
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relates to selected variables. For each of three reading tests , means 
» and standard deviations are reported separately for each form of 
administration and for the total group in Table 1. Also irxluded in 

Insert Table 1 about here 

Table 1 are reliability data for R/EAL. 

In order to Assess the effect of form of test administration on 
literacy performance, a one-way analysis of variance was performed 
using form of administration as the main effect. Post hoc comparisons 
utilizing a Scheffe test (Hays> 1973) were also computed. Table 2 

Insert Table 2 about here 

indicates an F significant beyond the .01 level. Schef fe"^ post hoc 
comparisons showed significant differences between the Standard English 
taped mean and the Spanish taped mean, but no other significant 
comparison. 

The relationship of literacy performance to selected variables 
was examined separately for each of the three forms of test administration. 
These intercorrelations are reported in Table 3. 

Insert Table 3 about here 

Discussion 

The determination of literacy utilising a test designed to measure 
practical, functional reading skills in real life situations 
was examined in terms of tost reliability and validity. Reliability 
for the test is at or above .90 for cr.ch of 
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the three conditions as reported in Table 1. Treating the relationship 
between R/EAL and the Stanford Achievement Test as a type of concurrent 
validity, correlations of .74 for the English tiped version and .61 for 
the Spanish taped version (Table 3). as well as the somewhat lower but 
significant coriela-tion for the English printed version, suggest that to 
a considerable extent R/EAL and the Stanford have considerable overlapping 
variance and measure many of the same residing skills. 

An appraisal of the extent of literacy in the target popula- 
tion is also of interest in this study. More meaningful than the raw 
score means Vnd standard deviations is the percentage of subjects in 
each group that were at or above mastery level, previously defined as 
passing at least 80 percent of the items. Although the percentage figures 
vary depending on the .form of administration, the data reveal that in no 
case did more than approximately 25 percent of any group demonstrate 

4 

„astery of functional reading -skills . A nm»ber of explanations for such 
^ snail percentage might be offered, but additional examination of program 
variables would need to be conducted to verify explanatiou: One possible 
explanatory factor my ba that those who had already mastered functional 
literacy had left the program and that those students still served by the 
program wore those who continued to have a need and thus would not test 
at mastery level. On the other hand, it might be that programs, servicing 
diverse and perhaps conflicting constituents and goals, were not accomplishing 
tho Boal of -eatery of basic Uteracy skills for its population. A somewhat 
less extreme explanation would take into account possible Intervening 
variables which may inhibit program effectiveness towards .chUving literacy, 
in any event, continued study looking at more specific aspects of the program 

o 13 
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Is recommended. 

Turning next to a consideration, of the effect of mode of test 
presentation on literacy performance, the Anova in Table 2 and the 
subsequent Schcffc^ comparisons reveal significant differences between 
English taped and'Spanish taped modes, but no other significant differences. 
However, a closer examination of data relating to entry level reading skills 
(RJSl) and current reading skills (Stanford) of the three groups (Table 1) suggests 
a marked difference in reading skills of the English and Spanish groups. 
The explanation that differences in literacy performance were due 
to initial reading differences and language interference rather than to 
mode of testing is more compelling. It is suggested that additional 
studies be conducted with bilingual groups and under more controlled 
conditions in order to investigate this question. Although Davis and 
Personke (1968) found no differences among young children, possible 
language interference affecting information processing among low reading 
achievement subjects continues to warrant additional investigation. 

The lack of significant differences between the taped and printed 
English is also revealing. Even though earlier studies had not found 
differences due to test modality, it was hypothesized that the non-test 
like atmosphere and more' motivating forn\at of the taped condition, as well 
as the elimination of processing of directions and questions via the printed 
word, would result in better performance by those subjects presented with 
this mode. Yet the finding of no significant difference did not sub- 
stantiate this hypothesis. This binding adds additional support to the 
lock of significance of teat modality on test performance of Ferris and 



14 



-12- 

BEST COPY AVAIUBLE 

Nichols (1968) and Rcmcr (1974) . . On the other hand, on site observations 
of subjects taking both forms and candid comments by students and 
teachers reflect greater interest and enthusiasm for taking a test 
which, on the face of it , docs not appear to ba a test. The lack of 
significant results*, however, does not lead to the abandonment of the 
investigation of alternative test administration modalities for low 
achieving young adults; rather it leads to continued examination of 
factors which may optimize testing procedures and conditions among adult 
populations . 

Finally a consideration of the results of the study leads 
to an analysis and interpretation of the relationship of selected 
variables and literacy performance. These analyses were computed 
separately for each of the three testing conditions. Looking at the 
data in Table 3, one Is impressed first with the fact that the number of 

« 

years a student has spent in 'school prior to entrance in the Job Corps 
program is, in all cases except one, significantly and positively related 
to performance on literacy measurements (R/EAL) , to performance on current 
measures of reading (Stanford Achievement Test), and to ijiltial performance 
on reading measures (RJSl) . These correlations, consistent with data re- 
ported by Stauffer (1973) , suggest that the more time a student has spent 
in school prior to entrance into a program, the more he will achieve in 
reading. This finding holds constant for the three separate groups. The 
consistency of the finding takes on additional meaning when mean number of 
years of schcrl completed under the English conditions (audio or printed) 
is comixired \rith that of the group receiving the Spanish tape. Although 
the former groups had completed an average of nine years of school and 

16 
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the latter group less than seven and a half, the positive relationship of 
years of school completed and reading performance is still confirmed. 
Consistent with these findings is the negative relationship (in one case 
significant) between the number of years of school completed and the 
length a student stays in the program. Precise reasons for this negative 
relationship are unclear, but possible explanatory factors might be less 
need for the program because of higher entry level ability, better chance 
of program success in shorter time period, or lack of congruence between 
progr&m goals and student needs. Whether or not the students who had 
completed more school remained in the program a shorter time because 
their needs were not as great, their goals were accouplished faster, or 
their needs remained unmet must be explored more fully in subsequent 
studies. 

Another characteristic examined in the study was the relation- 
ship of sex of the student to other variables. Correlations of the stu- 
dent^s sex to literacy performance revealed no significant correlations in 
any of the three comparisons. Although not significant, the data suggest a 
negative correlation between literacy and males for the English printed 
condition; conversely, the relationship w.us reversed for the Spanish faped 
condition in which being female was positively related to literacy. In 
the largest number of cases, though, nc relationship occurred. The incon- 
clusive evidence presented here substantiates the conflicting evidence re- 
ported earlier. 

Also explored in this Ft^dy was the student •s reading performance 
at the time of progrfim entry and tta relationship to literacy. In all cases, 

16 
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there was a significant positive relationship between entry level reading 
performance (RJSl) and subsequent measures of reading and literacy. This 
not unanticipated finding suggests that performacc in reading and literacy 
tasks at one time period is closely associated with measured reading 
ability at an earlier point in time. 

Contrary to Stauffer's finding about the relationship of pro- 
gram duration to success was the negative, but not significant, re- 
lationship between reading performance and the length a student remains 
in the program. At least three possible explanations for this lack of 
relationship may be offered; program goals were not directed at functional 
literacy and thus it would not be expected that those who remained in a 
program longej: would achieve greater literacy; program goals were directed 
at functional literacy but- were not successful; or program goals were 
directed at functional literacy but :hc measure used to assess literacy 
was unable to detect differences. Fu-^ther since those students who 
Initially were highest in reading ability stayed a shorter length of time 
than those who were lower, it rany be that length of time in the program 
is an irrelevant variable in explaning literacy achievement. 

As a whole it would appear that certain btudont characteristics, 
in particular the number of years of school completed and initial re^-'ding 
performance, are positively related to subsequent performance on reading 
and literacy tasks. Due to the limited population under study, it was not 
possible to explore additional "student characteristics, such as various 
homo and family background variables which have be 6 suggested as influencing 
reading performance (Coleman, 1966; Maycsi'-^.n.d.) . Subsequent studios with 
similar populations may shed additional .light on explanatory variables. 

17 
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Summary 

This research has examined literacy assessment from two 
perspectives: the effect of test administration format on literacy 
performance and the association of literacy performance with selected 
characteristics of students and programs. The most salient findings re- 
vealed significant differences between Spanish- taped and English -taped 
administration procedures. However, as suggested this difference is 
apparently due more to different characteristics of the samples 
(e.g. entry level reading, and years of school completed) than to form 
of administration. To further confirm this supposition an additional 
Study with more comparable groups should be undertaken. Other 
Important findin^P reveal significant positive relationships 
between literacy performance and number of years of school completed, 
entry level reading behavior, and current reading behavior. Moreover, 
no significant relationships between sex or length of stay in the program 
and literacy were observed. This study serves only as a beginning in 
explorations of literacy assessmtiit through new technique«. Currently 
studies assessing factors related to gros^th in literacy as measured by 
r/EAL are being conducted. A large study presently in prcsrcss identific 
and measures large numbers of student and program variables and their re- 
lationship to literacy growth. 
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Table 2 
Analysis of Variance 
* by Form of Administration 



Source df MS F 



Between groups 2 791.24 7.79* 

Within 622 101.62 

*p < .01 
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Intercorrclatlons of Selected Variables 
by Form of Administration 



English - Taped 

I. 2. 3. 4. 5.* 6. 

Sex Years RJSl Length Stanford R/EAL 
• School Program 

1. — .24* -.04 . -.13 . .08 . .02 

2. — .25* -.23* .28* .28* 

3. — -.14 .70* .65* 

4. — -.03 .07 

5. — -74* 
6. 

2. English - Printed 

1. 2. 3. 4. 5. 6. 

1. — .30 ' .09 -.18 .15 -.18 

" ■ til 

• *• 

2. — .37** -.24 .42* ^25 

3. — -.32 .44* .51* 

4. — - .15 -.18 

5. — .35** 
6. 

3. Spanish - Taped 

1. 2.' 3. 4. 5. 6. 

1. — .22 .06 .23 .14 .20 

2. — .38* -.10 .41* .39* 

3. — -.24 .67* .46* 

4. .19 .06 

5. — .61* 

6. 

21 

* Sig. beyond .01 level ** 
** SIk. beyond .05 level 
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